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Abstract — A random construction of bipolar sensing matrices 
based on binary linear codes is introduced and its RIP (Restricted 
Isometry Property) is analyzed based on an argument on the 
ensemble average of the weight distribution of binary linear 
codes. 

I. Introduction 

Research in compressed sensing 121 O is expanding rapidly. 
The sufficient condition for -recovery based on the Re- 
stricted Isometry Property (RIP) III ID is one of the celebrated 
results in this field. The design of sensing matrices with 
small RIP constants is a theoretically interesting and chal- 
lenging problem. Currently, random constructions provide the 
strongest results, and the analysis of random constructions is 
based on large deviations of maximum and minimum singular 
values of random matrices fS) JS). 

In the present paper, a random construction of bipolar 
sensing matrices based on binary linear codes is introduced 
and its RIP is analyzed. The column vectors of the proposed 
sensing matrix are nonzero codewords of a randomly chosen 
binary linear code. Using a generator matrix, a px m sensing 
matrix can be represented by 0(plog2 TO)-bits. The existence 
of sensing matrices with the RIP is shown based on an 
argument on the ensemble average of the weight distribution 
of binary linear codes. 



II. Preliminaries 



A. Notation 



The symbols M and F2 represent the field of real numbers 
and the finite field with two elements {0,1}, respectively. The 
set of all p X TO real matrices is denoted by M''^™. In the 
present paper, the notation x e M*^ indicates that x is a column 



vector of length p. The notation 
p < 00) defined by 




denotes ^„-norm (1 < 



(1) 



The £o-norm is defined by 



A 



cello = |supp(x)|, 



(2) 



where supp(x) denotes the index set of nonzero components of 
X. The functions Wh{-) and dh{-, ■) are the Hamming weight 
and Hamming distance functions, respectively. 



B. Restricted isometry property (RIP) 

Let $ = {01, . . . , 0m} G W^"^ be a p X TO real matrix, 
where the i'2-norm of the j-th (j G [1, to]) column vector 
is normalized to one, namely, ||0i||2 = 1- The notation [a,b] 
represents the set of consecutive integers from a to b. 

The restricted isometry property of $ introduced by Candes 
and Tao IS) plays a key role in a sufficient condition of £1- 
recovery. 

Definition 1: A vector x G M™ is called an S'-sparse {S E 
[1,to]) vector if ||x||o < 5*. If there exists a real number 
S{0 < S < 1) satisfying 



il-S)\\x\\l<\\^x\\l<{l + 6)\\x\\l 



(3) 



for any 5-sparse vector x E R"', then we say that $ has the 
RIP of order S. If $ has the RIP of order S, then the smallest 
constant satisfying ^ is called the RIP constant of $, which 
is denoted hy 63- □ 
Assume that $ has the RIP with small Ss- In such a case, any 
sub-matrix composed from Q-columns (1 < Q < S*) of $ is 
nearly orthonormal. 

Recently, Candes ||4l reported the relation between the RIP 
and the £i-recovery property. A portion of the main results of 
[|4j is summarized as follows. Let 5 G [1, to], and assume that 
$ has the RIP with 

^25<V2-L (4) 

For any 5-sparse vector e £ M™ (i.e., ||e||o < S), the solution 
of the following ^i-minimization problem 



ininimize||d||i subject to = s 



(5) 



coincides exactly with e, where s = $e. Note that lH consid- 
ers stronger reconstruction results (i.e., robust reconstruction). 
The matrix <& in (|5]) is called a sensing matrix. 

C. Relation between incoherence and the RIP 

The incoherence of $ defined below and the RIP constant 
are closely related. 

Definition 2: The incoherence of $ is defined by 

A 



^i{<^>) 



max |0f0j|. 



(6) 
□ 



The following lemma shows the relation between the inco- 
herence and the RIP constant. Similar bounds are well known 
(e.g.,11). 

Lemma 1: Assume that $ e is given. For any S E 

[1,to], Ss is upper bounded by 

Ss < K^)S. (7) 

An elementary proof (different from that in (91) is presented 
in Appendix. q 

III. Construction of sensing matrices based on 

BINARY LINEAR CODES 

In this section, we present a construction method for sensing 
matrices based on binary linear codes. A sensing matrix 
obtained from this construction has a concise description. A 
sensor can store a generator matrix of a binary linear code, 
instead of the entire sensing matrix. 

A. Binary to bipolar conversion function 

The function (3p : W" is called a binary to bipolar 

conversion function defined by 



/3 : x e F? ^(e- 2a;) G 



(8) 



where e is an all-one column vector of length p. Namely, using 
the binary to bipolar conversion function, a binary sequence 
is converted to a {+1/^, — l/^}-sequence. 

The following lemma demonstrates that the inner product 
of two bipolar sequences /3p(a) and Pp{b) is determined from 
the Hamming distance between the binary sequences a and b. 

Lemma 2: For any a,b G Fj, the inner product of f3p{a) 
and Pp{b) is given by 

2dh{a,b) 



(3p{ay (3p{b) = 1 



(9) 



(Proof) Let /3p(a) = (oi, . . . , Cp)^ and (3p{b) = . . . , bp)"^. 
Define Yi and I2 by 

Yi = {ie [l,p] : ai = bi}, Y2 = {i e [l,p] : at ^ bi}, 

(10) 

where \Yi\ ^ p — dh{a,b) and 1^21 — dh{a,b). Equation ^ 
is derived as follows: 

p 

(3p{af(3p{b) = ^a,6. 



p 



aibi + Qib. 

5^ n ^ ( r) 



P 



ieY2 



{p- dh{a,b))- +dh{a,b) [ 

p \ p 



1 



2dh{a,b) 



(11) 
□ 



It is easy to confirm that /3p(a) is normaUzed, i.e., | \Pp{a) \ \2 
1, for any a G F2. 



B. Construction of the sensing matrix 

Let if e F2 (p > r) be a binary rxp parity check matrix 
where 2^^"^ > p holds. The binary linear coded C{H) defined 
by H is given by 



C{H) = {x e¥l:Hx^ 0''}, 



(12) 



where C is a zero-column vector of length r. The following 
definition gives the construction of sensing matrices. 

Definition 3: Assume that all of the nonzero codewords of 
C{H) are denoted by ci, C2, . . . , cm (based on any predefined 
order), where M = 2?'~™"'=(^) - 1 > 2''-'' - 1. The sensing 
matrix e M^^™ is defined by 



A 



$(i/)-(/3p(ci),/?p(c2),...,/3p(c™))^ 



(13) 



where m = 2^-'' - 1. If $(if) has the RIP of order S, the 
RIP constant corresponding to $(if) is denoted by 5s{H). q 
Since the order of the columns is unimportant, we do not 
distinguish between sensing matrices of different column order 
(or choice of codewords from C{H)). 

If the weights of all nonzero codewords of C{H) are very 
close to p/2, then the incoherence of becomes small, 

as described in detail in the following lemma. 

Lemma 3: Assume that e(0 < e < 1) is given and that 



^ ' P<Wh{c) < ( )p (14) 



holds for any c S C{H)\Op. In such a case, the incoherence 
^{H) is upper bounded by 



(15) 



(Proof) For any pair of codewords a,b{a ^ b) E C{H), the 
Hamming weight of a + is in the range: 



due to the linearity of C{H). This means that 



p < dh{a,b) < — — ) p 



(16) 



(17) 



holds for any a,b E C{H){a ^ b). Using Lemma |2] we 
immediately obtain 

yi,jii^j)E[l,m], -e < Ppic^f Pp{cj) < e, (18) 

where 

= (/3p(ci), /3p(c2), . . . , /3p(c™)) . (19) 

The definition of incoherence and the above inequaUties lead 
to an upper bound on the incoherence: 



KHH)) < e. 



(20) 
□ 



C. Analysis based on ensemble average of weight distribution [3] Next, we evaluate the ensemble expectation of (H) 

We here consider binary linear codes whose weight dis- -^-l^^ 
tribution is tightly concentrated around the Hamming weight ^flr.p [-^e(^)] — ^Rr,p[-^w{H)] 



w — l 

p 



p/2. Before starting the analysis, we introduce the weight 

distribution {yl^(i/)}u,g[i „], which is defined by _l_ ^ ^^^j 



AUH) = \{c:ceCiH),Whic) = w}\. (21) l{i^)pi 



In the present paper, we consider an ensemble of binary parity ™~ w=\{^)p\ 

check matrices, which is referred to herein as the random [(i=-£)pj 

ensewWe. The random ensemble i?r.p contains all binary r xp <; 2^^'^ I- (27) 

i„! 1 Tt/ rT\ 1 Inn-) !_ ! J i_ 1- ' ^ ^ W ' 



matrices. Equal probability P{H) ~ 1/2^^ is assigned to each 

matrix in i?„ „. Let f be a real-valued function defined on i?„ „, rru c i • i . • j ..urn • -j .u 

The final mequality is due to the followmg identity on the 

which can be considered as a random variable defined over , . . , ■ ^ 

, , „ rr,, ■ r r ■ , . binomial coefficients: 

the ensemble it,.,p. The expectation of / with respect to the 

ensemble iir.p is defined by Vw G [0,p], (^^ ^ ( ^ \ (28) 



w/ \p — w 



Er [/] ^ P{H)f{H) (22) Using the Markov inequality, we obtain the following upper 

ifefi' bound on the probability of the event K^{H) > 1: 



Prob[K,{H) > 1] < Er^JK^IH)] 
The expectation of weight distributions with respect to the [(i^)pj 
random ensemble has been reported ID to be < 2^"' i^i 

w=0 



(29) 



Er [Aw(H)] = ( ^ ]2~^ (23) Since K^{H) takes a non-negative integer- value, we have 

VC-^)P\ . s 

In the following, a combination of average weight distribu- Prob[K^{H) 0] > 1 2 \w) ^^^^ 

tion and Markov inequality is used to show that the RIP holds 

for with overwhelmingly high probabiUty. This completes the proof. □ 

r AH .u. J vul . f The following theorem is the main contribution of the 

Lemma 4: Assume that we draw a parity check matrix from ° 

present paper 

Theorem 1: Assume that H is chosen 

to the probability assignment of Rr.p- If 



Rrp- The probability of selecting H that satisfies < present paper 

e is lower bounded by Theorem 1: Assume that H is chosen randomly according 



(Proof) Let us define K^{H) as 



1 ''t^'"(p\ S<zJ-J^, (31) 

holds, then S2s{H) < \/2 — 1 holds with probability greater 
than 

1 - 2^~P+'', (32) 



L(i^)pj p where m = 2^ — 1. The constant Z is given by 

K,iH)= E ^u.{H)+ j2 ^-(^) (25) aV2-1 

^=1 ^o=[{^p)p] ^^^^ 



for H E Rr,p- The condition K^{H) = implies that 



(Proof) A simpler upper bound on 



P < Whic) < —— p (26) „,=o 



T. (:) (34) 



for any c G C{H)\Op. Namely, if /^^(i/) = holds, then 
fi{^{H)) is proven to be smaller than or equal to e by Lemma 



is required. Using the inequality on binomial coefficients (|6l: 

^ ) < 2P^("'/P\ (35) 



we have 



t«=0 



■u;=0 



< 21-'' X p X 2^'^(^) 

where is the binary entropy function defined by 

H{x) = -xlog2 a; - (1 - x) log2(l - x). (37) 

In order to consider the exponent of an upper bound, we 
take the logarithm of (|34] | and obtain an upper bound of the 
exponent: 



l0g2 



L(t^)pJ 



< l+log2(m + l)-p + log2P 

1 - 



pH 



(38) 



1 



In the above derivation, we used the relation 

r = p — logj (m + 1 ) 



(40) 



and the assumption 2p~^ > P- A quadratic upper bound on 
the binary entropy function (Lemma |6] in Appendix) was also 
exploited to bound the entropy term. 
Letting 



A /61og2(m + l) 



(41) 



we have 



l + 21og2(m + l)- -pe 



2 = l-log2(m + l) 

= 1-p + r. (42) 



Lemma [Hand Lemma |4] imply that, in this case, Ss{H) < eS 
holds with probability greater than 1 — 2^-^+''. Due to Lemma 
[T] the £i-recovery condition ^ can be written as 



From this inequality, we have 
S <Z 



(43) 



P 



< Z 



P 



log2(m + 1) Y log2w' 
which proves the claim of the theorem. 



(44) 
□ 



D. Asymptotic analysis 

In this subsection, the asymptotic properties of the proposed 
construction are given. 

Lemma 5: Assume that we draw a parity check matrix from 
Rr,p- The probability of selecting H that satisfies < 
e is upper bounded by 



(1- 2-021+- Eilj 



(45) 



(Proof) Here, we use a variant of Chebyschev's inequality |[T|: 

Prob[Km-0]< . (46) 

where VARr^ (•) denotes the variance with respect to Rr,p- 
The variance VARr^^{K^{H)) is given by 

VARn^jKAH)) 

A A A p 

= Cov{wi,W2) + ^ ^ Cov{wi,W2) 

tWl — 1UJ2 — 1 Wl — lw2—B 

p A P P 

+y~^ cov{wi,w2) + y^ y^ cow(wi, 102X47) 

-u;!— _Bu;2 — 1 Wi—Bw2 — B 

where ^ = [(1 — e)p/2j and S = [(H-e)p/2] . The covariance 
of weight distributions denoted by Cov{wi,'W2) is defined as 
follows: 

Cov{wi,W2) = Er^JA^AH)A^AH)] 

- Er^JA^,{H)]Er^JA.^AH)] (48) 

for Wi,W2 £ [IjJ^]- The covariance for the random ensemble 
has the following closed formula ifTOl : 



C0V[Wl,W2) = S \ ' \w/ 

10 Wi i= W2 



W 



(49) 

for wi,W2 e [i,n]. Applying the covariance formula to (07]), 
we have 

VARr^JKAH)) 

\w = l ^ ' w=B ^ 

< (l-2-'^)2i-X:(^)- (50) 
Plugging the expectation of Ke{H) 

^ ( ^ ( w ) ^ ^ ( w 

= l V / w=B ^ 



ErIKAH)] 



2-'- 2^ 



1 



(51) 



and the upper bound on the variance ( fSOl l into ( l46b proves the 



lemma. 



□ 



The asymptotic behavior of Proh[K^{H) = 0] and 
Prob\K^{H) 7^ 0] is summarized in the following theorem. 

Theorem 2: Assume that a = r/p \s fixed (0 < a < 1). 
Let 

/i(e,a) = \im-\og^Prob[K,{H)=Q\ (52) 
f2{e,a) = lim-\og2Prob[K,{H)^0]. (53) 

p— *oo p 

The following inequalities give upper bounds on /i(e) and 
/2(e), respectively: 

(54) 
(55) 

(56) 
(57) 



/i(e, a) < a-H 

/2(e,a) < -a + H 
(Proof) We first discuss 



2 

1 - e 



X 



A 



Let 

L(l-e)p/2j 

E 



Using the inequality on the binomial coefficients 

P\ ^ 1 



J {p + 1)2 
X can be bounded from below: 



2pH{w/p) 



X > 



> 



P 

L(i - fW2J 
1 



2pH((l-€)/2-l/p)_ 



(p+l)2 

The inequality (|45] | can be simplified as 

(1 _ 2-°'P)2^+°'PX 



(58) 



(59) 



(2X- 1)2 

for sufficiently large X. The right-hand side of the above 
inequality can be bounded from above using (BFl i: 

2l+apj^-l < + ;L)22l+ap-p//((l-0/2-l/p)_ (gQ) 

We are now able to derive the inequality given in (|54] i as 
follows: 



lim - log2 

p — voo J> 



(p + l)22l+"P-P^((l-0/2-l/p) 

1 - e 



(61) 



The inequality given in i55[ is readily obtained from (l38T l. q 
Theorem|2]implies a sharp threshold behavior in the asymp- 
totic regime. Let a*{e) be 

'1 - 



A 



a*{e) = H 



(62) 



which is referred to as the critical exponent. If a < Q!*(e), 
(|54] | means that the probability to draw a p x r matrix with 
< e decreases exponentially as p goes to infinity. 
On the other hand, (l55l l indicates that the probability not to 
select a matrix with < e decreases exponentially if 

a > a*{e). 



IV. Concluding remarks 

In the present paper, a construction of a bipolar sensing 
matrix is introduced and its RIP is analyzed. The existence 
of sensing matrices with the RIP has been shown based on a 
probabilistic argument. An advantage of this type of sensing 
matrix is its compactness. A sensor requires 0(pm)-bits in 
order to store a truly random p x m bipolar matrix. On the 
other hand, we need only 0(plog2 m)-bits to store 
because we can use a generator matrix of C (H) as a compact 
representation of C{H). However, this limited randomness of 
matrices results in a penalty on the RIP constant. Although the 
present construction is based on a probabilistic construction, 
the results shown in Theorem [T] are weaker than the £1- 
recovery condition 0(5 log^ (771/5*)) < p for the truly random 
p X m bipolar matrix ensemble shown in Q. The condition 
shown in Theorem [T] can be written as 0{S y/log^rn) < y/p 
and is more similar to the conditions of deterministic construc- 
tions, such as that given in I?]. 

Lemma [3] may be useful for evaluating the goodness of a 
randomly generated instance. The weight distribution of C{H) 
can be evaluated with time complexity 0{mp), and an upper 
bound on the RIP constant can be obtained using Lemma [3] 



Appendix 
Lemma 6: The following inequalitjQ 



2(.T--) >H{x)-l 



(63) 



holds for < .T < 1. 
(Proof) Let f{x) be 



(64) 



the domain of which is < a; < 1. The first and second 
derivatives of f{x) are given by 

fix) = -4ix- 1/2) - log2(l -x)+ log2 X (65) 



and 



1 1\ 1 



1 — X X I log^ 2 



(66) 



respectively. It is easy to verify that f"{x) > for < a; < 1, 
which indicates that f{x) is convex. Thus, we can obtain the 
global minimum of f{x) by solving f'{x) = 0, and we have 

/'(1/2) = and / (1/2) = 0. □ 

'This bound becomes tighter as x approaches to 1/2. 



Proof of Lemma Q] [6] T.M.Cover and J.A.Thomas, "Elements of Information Theory", 2nd ed. 

„ . ... „ ^ \ \n\ ^ Q Wiley-Interscience 2006. 

Let Q be an index set satlStying Q C |1, . . . , mj, \ Q\ < b. [7] r a. DeVore, "Deterministic constructions of compressed sensing ma- 
For any C = {ci)i(zQ S RI'^I, we have trices, " J. of Complexity, 23, pp. 918 - 925, 2007. 

_ [8] R.G.Gallager, "Low Density Parity Check Codes". Cambridge, MA:MIT 

Press 1963. 

2 , \T t ^ \ I J. 1 / J. 1 H.Rauhut, K.Schass, and P.Vandergheynst, "Compressed sensing and 

redundant dictionaries," IEEE Trans, on Information Theory, vol. 54(5), 
pp. 2210 - 2219, 2008. 

^ 2 ^ [10] T.Wadayama, "On undetected error probability of binary matrix ensem- 

— / / (^i'^j^'i 4'j ~ / + y CiCj(p^ (f>j bles", in Proceedings of IEEE International Symposium on Information 
ieQj£Q ieQ i jeQii^j) Theory (ISIT2008), Tronto (2008) (related preprint: arXiv:0705.3995j . 

< ^c2 + ^($) (67) 

where <i>Q is a sub-matrix of composed from the columns 
corresponding to the index set Q. For any a, 5 e R, 

{a^ +b^)/2>\ab\ (68) 

holds since (|a| - \b\f = + 6^ _ 2|a6| > 0. We use this 
inequality to bound \ciCj\ in ( |67] | and obtain 

< 




ieQ ieQ 

= \\c\\l{^ + ^^mQ\) 

< ||c||2(l + ^($)5). 



Similarly, ||$qc||| can be lower bounded by ||$qc||2 > 
||c||2(l — fj,{^)S). From the definition of 5s, the lemma is 
proven. 



ACKNOWLEDGMENT 

The author would like to thank the anonymous reviewers of 
IEEE Information Theory Workshop 2009 for their construc- 
tive comments. The present study was supported in part by the 
Ministry of Education, Science, Sports and Culture of Japan 
through a Grant-in-Aid for Scientific Research on Priority 
Areas (Deepening and Expansion of Statistical Informatics) 
180790091. 



References 

[1] N.Alon and J.H.Spencer, "The probabilistic method, " 2nd ed., John 
Wiley & Sons, 2000. 

[2] E.Candes, J. Romberg and T.Tao, "Robust uncertainty principles: exact 
signal reconstruction from highly incomplete frequency information," 
IEEE Trans, on Information Theory, vol.52(2), pp. 489 - 509, 2006. 

[3] E.Candes and T.Tao, "Decoding by linear programming, " IEEE Trans, 
on Information Theory, vol.51(12), pp. 4203 - 4215, 2005. 

[4] E.Candes, "The restricted isometry property and its implications for 
compressed sensing," Compte Rendus de I'Academie des Sciences, 
Paris, Serie I, pp. 589-592. 

[5] E.Candes and T.Tao, "Near optimal signal recovery from random pro- 
jections: universal encoding strategies?, " IEEE Trans, on Information 
Theoiy, vol.52(12), pp. 5406 - 5425, 2006. 



